Novel HPC Technologies for Scalable CAE: The Case for Parallel I/O and File Systems
نویسنده
چکیده
As HPC continues its aggressive platform migration from proprietary supercomputers and Unix servers to HPC clusters, expectations grow for clusters to meet the I/O demands of increasing fidelity in CAE modeling and data management in the CAE workflow. Cluster deployments have increased as organizations seek ways to costeffectively grow compute resources for CAE applications, and during this migration many also implemented conventional network attached storage (NAS) architectures to simplify IT administration and further reduce costs. While legacy NAS implementations offer several advantages of shared file systems, most are too limited in scalability for effective management of I/O demands with parallel CAE applications. As such, a new storage migration is underway to replace legacy (serial) NAS with parallel NAS architectures and parallel file systems. This new class of parallel file system and shared storage technology was developed to scale I/O in order to extend the overall scalability of CAE simulations on clusters. This paper examines CAE motivation for shared parallel file systems and storage, for requirements of multi-physics LS-DYNA applications on conventional clusters with proper balance for I/O. Model parameters such as size, element types, schemes of implicit and explicit (and coupled), and a variety of simulation conditions can produce a wide range of computational behavior and I/O data management demands. The benefits of a Panasas storage implementation are introduced for such broad requirements, through examples of CAE workflows for a variety of production-level applications in industry.
منابع مشابه
Tuning HDF5 for Lustre File Systems
HDF5 is a cross-platform parallel I/O library that is used by a wide variety of HPC applications for the flexibility of its hierarchical object-database representation of scientific data. We describe our recent work to optimize the performance of the HDF5 and MPI-IO libraries for the Lustre parallel file system. We selected three different HPC applications to represent the diverse range of I/O ...
متن کاملScaling Security for Big, Parallel File Systems
The need for petaand exabyte scale parallel file systems that support high-performance computing (HPC) has been rapidly increasing. These systems have unique demands, different from those of traditional distributed file systems. As a result, securing I/O in big, parallel file systems without significantly impacting performance has proven challenging. Parallel file systems are commonly composed ...
متن کاملpNFS and Linux: Working Towards a Heterogeneous Future
Heterogeneous and scalable remote data access is a critical enabling feature of widely distributed collaborations. Parallel file systems feature impressive throughput, but sacrifice heterogeneous access, seamless integration, security, and cross-site performance. Remote data access tools such as NFS and GridFTP provide secure access to parallel file systems, but either lack scalability (NFS) or...
متن کاملvPFS: Bandwidth Virtualization of Parallel Storage Systems
This paper presents vPFS, a new parallel file system performance management approach to support the allocation of shared storage bandwidth on a per-application basis. Existing parallel file systems are unable to differentiate I/O requests from different applications and meet per-application bandwidth requirements. This limitation presents an increasing hurdle for applications to achieve their d...
متن کاملThe Edge Node File System: A Distributed File System for High Performance Computing
The concept of using Internet edge nodes for High Performance Computing (HPC) applications has gained acceptance in recent times. Many of these HPC applications also have large I/O requirements. Consequently, an edge node file system that efficiently manages the large number of files involved can assist in improving application performance significantly. In this paper, we discuss the design of ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2008